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I. REAL PARTY IN INTEREST 

The real party in interest is Koninklijke Philips Electronics N.V., 
Eindhoven, The Netherlands by virtue of an assignment recorded June 13, 
2006 at reel 017773, frame 0794. 

II. RELATED APPEALS AND INTERFERENCES 

There are no related appeals or interferences. 

III. STATUS OF CLAIMS 

This application was originally filed with Claims 1-14. Claims 1- 
14 stand finally rejected by the Office action of August 15, 2011 and are 
the subject of this appeal. 

IV. STATUS OF AMENDMENTS 

No amendments or other filings were submitted in response to the 
final rejection mailed August 15, 201 1 . A notice of appeal was timely 
filed on November 1, 201 1. 
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V. SUMMARY OF THE CLAIMED SUBJECT MATTER 

The present invention is a medical system which utilizes an x-ray 
system to acquire a two-dimensional (2D) x-ray image of a body with a 
medical instrument in the body. An ultrasound system has an ultrasound 
probe which acquires three-dimensional (3D) ultrasound data of the 
medical instmment. A localizer provides localization of the location of 
the ultrasound probe within the referential coordinate space of the x-ray 
system. A selector selects a region of interest around the medical 
instrument in the 3D ultrasound data set to define a first localization of 
the region of interest within the referential of the ultrasound system. A 
converter converts the first localization of the ultrasound system into a 
second localization of the region within the referential of the x-ray 
system, using the localization of the ultrasound probe. With this 
registration of the localizations of the two systems, a bi-modal 
representation of the medical instrument is displayed in which the 2D x- 
ray image and the 3D ultrasound data of the region of interest are 
combined using the second localization to provide a bi-modal image 
which retains the advantages of both imaging modalities. 

Claims 1 and 7 are supported by the drawings and specification as 
seen by reference numerals (#) of the drawings and the specification text 
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by page and paragraph number (pg., para) of the filed application as 
follows: 

1 . A medical system comprising: 

a medical instrument {#4; pg. 7, In. 5-7} to be guided in a patient 

body, 

an X-Ray acquisition system {#5,#6,#7; pg. 7, In. 7-12} which 
acquires a two-dimensional X-ray image of said medical instrument, 

an ultrasound acquisition system {#8,#9; pg. 7, In. 13-18} for 
acquiring a three-dimensional ultrasound data set of said medical 
instrument using an ultrasound probe, 

a localizer {#11; pg. 7, In. 25-26} which provides a localization of 
said ultrasound probe within a referential of said X-ray acquisition 
system, 

a selector {#12; pg. 7, In. 26-29} operable to select a region of 
interest around said medical instmment in the three-dimensional 
ultrasound data set, that defines a first localization of said region of 
interest within a referential of said ultrasound acquisition system, 

a convertor {#13; pg. 7, In. 29-31} which converts said first 
localization of said region of interest within said referential of the 
ultrasound acquisition system into a second localization of said region of 
interest within said referential of the X-ray acquisition system, using said 
localization of the ultrasound probe, 

a generation and display screen {#15; pg. 7, In. 31-34} which 
displays a bi-modal representation of said medical instrument in which 
said two-dimensional X-ray image and the three-dimensional ultrasound 
data included in said region of interest are combined using said second 
localization. 

14. A method of guiding a medical instrument in a patient 
body, comprising the steps of: 
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acquiring a two-dimensional X-ray image of said medical 
instrument using an X-ray acquisition system, {#60; pg. 14, In. 19-20} 

acquiring a three-dimensional ultrasound data set of said medical 
instrument using said ultrasound probe and an ultrasound acquisition 
system, {#61; pg. 14, In. 21-22} 

localizing said ultrasound probe in a referential of said X-ray 
acquisition system, {#62; pg. 14, In. 23} 

selecting a region of interest of said medical instmment within 
said 3D ultrasound data set, that define a first localization of said region 
of interest within a referential of said ultrasound acquisition system, 
{#63; pg. 14, In. 24-26} 

converting said first localization within said referential of said 
ultrasound acquisition system into a second X-Ray localization within 
said referential of the X-ray acquisition system, {#64; pg. 14, In. 27-28} 

generating and displaying a bimodal representation of said 
medical instrument in which said two-dimensional X-ray image and the 
three-dimensional ultrasound data included in said region of interest are 
combined using said second localization. {#65; pg. 14, In. 29-31} 

VI. GROUNDS OF REJECTION TO BE REVIEWED 

ON APPEAL 

A. Whether Claims 1-13 were properly rejected under 35 
U.S.C. §103(a) as being unpatentable over US Pat. 6,574,492 (Ben-Hiam 
et al.) in view of US Pat. 6,233,477 (Chia et al.) in view of US Pat. 
5,357,550 (Asahina et al.) 

B. Whether Claim 14 was properly rejected under 35 U.S.C. 
§ 103(a) as being unpatentable over Ben-Hiam et al. in view of Chia et al. 
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in view of Asahina et al. in view of US pat. pub. no. US2004/02 54454 
(Kockro) 

VII. ARGUMENT 

A. Whether Claims 1-13 were properly rejected under 35 
U.S.C. §103(a) as being unpatentable over US Pat. 6,574,492 (Ben- 
Hiam et al.) in view of US Pat. 6,233,477 (Chia et al.) in view of US 
Pat. 5,357,550 (Asahina et al.) 

Amended Claim 1 describes a medical system comprising a 
medical instmment to be guided in a patient body, an X-Ray acquisition 
system which acquires a two-dimensional X-ray image of said medical 
instrument, an ultrasound acquisition system for acquiring a three- 
dimensional ultrasound data set of said medical instmment using an 
ultrasound probe, a localizer which provides a localization of said 
ultrasound probe within a referential of said X-ray acquisition system, a 
selector operable to select a region of interest around said medical 
instmment in the three-dimensional ultrasound data set, that defines a 
first localization of said region of interest within a referential of said 
ultrasound acquisition system, a converter which converts said first 
localization of said region of interest within said referential of the 
ultrasound acquisition system into a second localization of said region of 
interest within said referential of the X-ray acquisition system, using said 
localization of the ultrasound probe, and a generation and display device 
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which displays a bi-modal representation of said medical instrument in 
which said two-dimensional X-ray image and the three-dimensional 
ultrasound data included in said region of interest are combined using 
said second localization. An implementation of the present invention 
relates a two dimensional X-ray image to the coordinates of a three 
dimensional ultrasound image including a medical instmment and 
produces a bi-modal representation of the 2D X-ray image in the three 
dimensional ultrasound data with the medical instrument represented in 
the bi-modal representation. 

Ben-Hiam et al. describes a system for measuring physiological 
signals inside the chambers of the heart with two catheters. One catheter 
is inserted with its tip at a reference location such as the apex of the heart. 
The other catheter is then manipulated to make the measurements. Each 
catheter has three sensors at its tip so that their coordinates can be sensed 
with a field generator. The position of the working catheter can then be 
known in relation to the coordinates of the reference catheter. At the very 
end of the description Ben-Hiam et al. say that the positions of the 
catheters in relation to the heart can be visualized with X-ray or 
ultrasound. How this is done is not explained. There is also no use of 
three dimensional ultrasound by Ben-Hiam et al. No bi-modal 
representations are made by Ben-Hiam et al. 

8 



APPEAL 
Serial No.: 10/596,434 
Docket# 2003P03001WOUS 

Chia et al. describe a pair of catheters with ultrasound crystals at 
their tips which serve as beacons. This enables the tip of one catheter to 
do ranging with the other catheter. No imaging at all is done by Chia et 
al., only 3D ultrasound locating with the ranging system. No bi-modal 
representations are made by Chia et al., either. 

Asahina et al. are doing x-ray fluoroscopy and ultrasound 
imaging. Asahina et al. are not using 3D ultrasound, as made clear by 
their references to "frame memories" for the 2D image frames. 
Ultrasound images and fluoroscopy images are shown on separate 
displays, but time-matched so that concurrently acquired fluoroscopy and 
ultrasound images can be retrieved and shown at the same time. Asahina 
et al. are not forming bimodal images, either. 

It is thus seen that this combination of three patents lacks at least 
three dimensional ultrasound data sets and bimodal images which 
combine a 2D X-ray image with a three dimensional ultrasound data set. 
A selector which selects a location around a medical instrument in 3D 
ultrasound data is also not found in any of these patents. It is therefore 
respectfully submitted that the combination of these three patents cannot 
render Claim 1 or its dependent Claims 2-13 unpatentable. 
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B. Whether Claim 14 was properly rejected under 35 U.S.C. 
§103(a) as being unpatentable over Ben-Hiam et al. in view of Chia et 
al. in view of Asahina et al. in view of US pat. pub. no. 
US2004/0254454 (Kockro) 

Claim 14 was rejected under 35 U.S.C. Sec. 103(a) as being 

unpatentable over Ben-Hiam et al. in view of Chia et al. in view of 
Asahina et al. in view of US Pat. pub. 2004/0254454 (Kockro). Claim 14 
describes a method of guiding a medical instrument in a patient body, 
comprising the steps of acquiring a two-dimensional X-ray image of said 
medical instmment using an X-ray acquisition system, acquiring a three- 
dimensional ultrasound data set of said medical instrument using said 
ultrasound probe and an ultrasound acquisition system, localizing said 
ultrasound probe in a referential of said X-ray acquisition system, 
selecting a region of interest of said medical instrument within said 3D 
ultrasound data set, that define a first localization of said region of 
interest within a referential of said ultrasound acquisition system, 
converting said first localization within said referential of said ultrasound 
acquisition system into a second X-Ray localization within said 
referential of the X-ray acquisition system, and generating and displaying 
a bimodal representation of said medical instmment in which said two- 
dimensional X-ray image and the three-dimensional ultrasound data 
included in said region of interest are combined using said second 
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localization. The inventive method produces a bimodal representation of 
a 2D X-ray image in three dimensional ultrasound data with the medical 
instrument represented m its proper spatial location in the ultrasound 
data. 

Kockro describes a system which produces a heads-up display in 
front of a surgeon in which the surgeon can visualize CT or MRI images 
of a surgical site such as the head. Using a probe 9, the surgeon can 
manipulate the data in space. Like the other three references, Kockro 
lacks any use of 3D ultrasound data and does not show or suggest 
bimodal representations of a 2D X-ray image in a 3D ultrasound dataset. 
The Examiner says that a "bounding box" constitutes selecting the region 
of interest of a medical instrument, but a reading of the cited passage says 
that the bounding box only delineates the surgical site, in this patent, 
surrounding the head. A surgical instrument could be anywhere in the 
head and is thus not spatially defined by the box. For all of these reasons 
it is respectfully submitted that Claim 14 is patentable over these four 
patents. 
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VIII. CONCLUSION 

Based on the law and the facts, it is respectfully submitted that 
Claims 1-13 are patentable over the combination of Ben-Hiam et aL, Chia 
et al. and Asahina et al. and that Claim 14 is patentable over the 
combination of Ben-Hiam et al., Chia et al., Asahina et al. and Kockro. 
Accordingly, it is respectfully requested that this Honorable Board 
reverse the grounds of rejection of Claims 1-14 of this application which 
were stated in the August 15, 201 1 Office action being appealed. 

Respectfully submitted, 

OLIVIER GERARD ET AL. 

Bv: /W. Brinton Yorks. Jr./ 
W. Brinton Yorks, Jr. 
Reg. No. 28,923 
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APPENDIX A: CLAIMS APPENDIX 
The following Claims 1-14 are the claims involved in this 

appeal. 

1 . (previously presented) A medical system comprising: 

a medical instrument to be guided in a patient body, 

an X-Ray acquisition system which acquires a two-dimensional 
X-ray image of said medical instmment, 

an ultrasound acquisition system for acquiring a three- 
dimensional ultrasound data set of said medical instrument using an 
ultrasound probe, 

a localizer which provides a localization of said ultrasound probe 
within a referential of said X-ray acquisition system, 

a selector operable to select a region of interest around said 
medical instmment in the three-dimensional ultrasound data set, that 
defines a first localization of said region of interest within a referential of 
said ultrasound acquisition system, 

a convertor which converts said first localization of said region of 
interest within said referential of the ultrasound acquisition system into a 
second localization of said region of interest within said referential of the 
X-ray acquisition system, using said localization of the ultrasound probe, 

a generation and display screen which displays a bi-modal 
representation of said medical instmment in which said two-dimensional 
X-ray image and the three-dimensional ultrasound data included in said 
region of interest are combined using said second localization. 
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2. (previously presented) A system as claimed in claim 1, 
wherein said selector is operable to select a region of interest intended to 
define a reference plane in which a part of said medical instrument is 
included. 

3. (original) A system as claimed in claim 2, wherein said region 
of interest is a 2D ultrasound image obtained by sampling said 3D 
ultrasound data set over said reference plane. 

4. (original) A system as claimed in claim 2, wherein said region 
of interest is obtained by cropping a 3D ultrasound data subset, which lies 
behind said reference plane or by cropping a slab which is formed around 
said reference plane. 

5. (previously presented) A system as claimed in claim 4, 
wherein said generation and display screen is intended to generate a 
volume rendered view of said region of interest within said 3D ultrasound 
data set. 

6. (previously presented) A system as claimed in claim 1, 
wherein said localizer is intended to localize an active localizer, which 
has been arranged on said ultrasound probe. 

7. (previously presented) A system as claimed in claim 1, 
wherein said ultrasound probe is equipped with at least three non aligned 
and interdependent radio -opaque markers and 

said localizer is intended to localize said markers in at least a first 
2D X-ray image having a first orientation angle in said referential. 
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8. (previously presented) A system as claimed in claim 1, 
wherein said localizer is intended to further localize said markers in a 
second 2D X-ray image having a second orientation angle in said 
referential. 

9. (previously presented) A system as claimed in claim 1, 
wherein said selector is operable to detect said medical instrument within 
said region of interest of the 3D ultrasound data set and 

said generation and display screen is intended to give to the points 
of the detected medical instmment in said bimodal representation the X- 
ray intensity values of the corresponding points in the 2D X-Ray image. 

10. (previously presented) A system as claimed in claim 1, 
comprising 

a segmenter operable to segment a wall tissue region in the 3D 
ultrasound data set and 

said generation and display screen is intended to give to the points 
belonging to said wall tissue region the ultrasound intensity values of the 
corresponding points of said region of interest. 

1 1 . (previously presented) A system as claimed in claim 1 , 
wherein the X-Ray acquisition system is intended to provide live two- 
dimensional X-Ray images and the ultrasound acquisition system live 
three-dimensional ultrasound data sets. 

12. (previously presented) A system as claimed in claim 1 \, 
comprising a controller for periodically triggering the probe localizer. 
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13. (previously presented) A system as claimed in claim 1 1, 
comprising 

a compensator operable to compensate a motion between a 
current three-dimensional ultrasound data set acquired at a current time 
and a previous three-dimensional ultrasound data set acquired at a 
previous time. 

14. (original) A method of guiding a medical instrument in a 
patient body, comprising the steps of: 

acquiring a two-dimensional X-ray image of said medical 
instrument using an X-ray acquisition system, 

acquiring a three-dimensional ultrasound data set of said medical 
instrument using said ultrasound probe and an ultrasound acquisition 
system, 

localizing said ultrasound probe in a referential of said X-ray 
acquisition system, 

selecting a region of interest of said medical instrument within 
said 3D ultrasound data set, that define a first localization of said region 
of interest within a referential of said ultrasound acquisition system, 

converting said first localization within said referential of said 
ultrasound acquisition system into a second X-Ray localization within 
said referential of the X-ray acquisition system, 

generating and displaying a bimodal representation of said 
medical instmment in which said two-dimensional X-ray image and the 
three-dimensional ultrasound data included in said region of interest are 
combined using said second localization. 
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APPENDIX B: EVIDENCE APPENDIX 



None. No extrinsic evidence has been submitted in this case. 
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APPENDIX C: RELATED PROCEEDINGS APPENDIX 



None. There are no related proceedings. 
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